Multiscale Structure Detection and Its Application to Image Segmentation and Motion Analysis

The problem of structure detection in images involves the identification of local groups of pixels that are both homogeneous and dissimilar to all nearby areas. Homogeneity can be measured with respect to any criteria of interest, such as color, texture, motion, or depth. No prior knowledge is assumed regarding the number of structures, their size or shape, or the degree of homogeneity that they must possess. Only the homogeneity criteria of interest have to be known. Structures may be either connected (pixels form contiguous areas) or disconnected, but the former case is treated in detail by this thesis. Structure identification is inherently a multiscale problem. For example, a texture contains subtexture, which itself contains subtexture, etc. In the absence of prior information, an algorithm must identify all such structures present, regardless of the scale. A formulation of scale is given that is able to describe image structures at different scales. A nonlinear transform is presented that has the property that it makes structure information at a given scale explicit in the transformed domain. This property allows the processes of automatic scale selection and structure identification to be integrated and performed simultaneously. Structures that are stable (locally invariant) to changes in scale are identified as being perceptually relevant. The transform can be viewed as collecting spatially distributed evidence for edges and regions and making it available at contour locations, thereby facilitating integrated detection of edges and regions without restrictive models of geometry or homogeneity variation. An application of this structure identification to the problem of estimating 2-D motion fields from video sequences is given. This approach has advantages in being able to compute accurate motion near occlusion boundaries and in areas with little variation in intensity.